Temporarily Disable Java Integration Tests #4957
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tustvold
force-pushed
the
integration-test-dependencies
branch
from
a6f87f7 to
cab4411
Compare
tustvold
added
the
development-process
|
@pitrou Any ideas, this at least appears to coincide with apache/arrow#38248 |
|
Right, you will also need to add this line to the build steps: and make sure you pass these env vars: |
|
Is there perhaps some way that we can script these in a way that can be reused by arrow-rs as opposed to hard-coded in a docker-compose file, that we keep having to incorporate changes from as things change? |
|
That's a good question. I'm not sure how to do that as these environment variables are influencing build steps for various implementations. Perhaps we need a single overarching environment variable such as |
|
How about adding diff --git a/docker-compose.yml b/docker-compose.yml
index e54c609e5..6b5b777c9 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -1734,18 +1734,8 @@ services:
# Tell Archery where the arrow C++ binaries are located
ARROW_CPP_EXE_PATH: /build/cpp/debug
ARROW_GO_INTEGRATION: 1
- ARROW_JAVA_CDATA: "ON"
- JAVA_JNI_CMAKE_ARGS: >-
- -DARROW_JAVA_JNI_ENABLE_DEFAULT=OFF
- -DARROW_JAVA_JNI_ENABLE_C=ON
command:
- ["/arrow/ci/scripts/rust_build.sh /arrow /build &&
- /arrow/ci/scripts/cpp_build.sh /arrow /build &&
- /arrow/ci/scripts/csharp_build.sh /arrow /build &&
- /arrow/ci/scripts/go_build.sh /arrow &&
- /arrow/ci/scripts/java_jni_build.sh /arrow $${ARROW_HOME} /build /tmp/dist/java/$$(arch) &&
- /arrow/ci/scripts/java_build.sh /arrow /build /tmp/dist/java &&
- /arrow/ci/scripts/js_build.sh /arrow /build &&
+ ["/arrow/ci/scripts/integration_build.sh /arrow /build &&
/arrow/ci/scripts/integration_arrow.sh /arrow /build"]
################################ Docs #######################################diff --git a/.github/workflows/integration.yml b/.github/workflows/integration.yml
index 62d2d2cb1a..0c3e4213fd 100644
--- a/.github/workflows/integration.yml
+++ b/.github/workflows/integration.yml
@@ -88,42 +88,10 @@ jobs:
with:
path: rust
fetch-depth: 0
- - name: Install pythonnet
- run: conda run --no-capture-output pip install pythonnet
- - name: Install archery
- run: conda run --no-capture-output pip install -e dev/archery[integration]
- - name: Make build directory
- run: mkdir /build
- - name: Build Rust
- run: conda run --no-capture-output ci/scripts/rust_build.sh $PWD /build
- - name: Build C++
- run: conda run --no-capture-output ci/scripts/cpp_build.sh $PWD /build
- - name: Build C#
- run: conda run --no-capture-output ci/scripts/csharp_build.sh $PWD /build
- - name: Build Go
- run: conda run --no-capture-output ci/scripts/go_build.sh $PWD
- - name: Build Java
- run: conda run --no-capture-output ci/scripts/java_build.sh $PWD /build
- - name: Build JS
- run: conda run --no-capture-output ci/scripts/js_build.sh $PWD /build
+ - name: Build
+ run: conda run --no-capture-output ci/scripts/integration_build.sh $PWD /build
- name: Run integration tests
- run: |
- conda run --no-capture-output archery integration \
- --run-flight \
- --run-c-data \
- --run-ipc \
- --with-cpp=1 \
- --with-csharp=1 \
- --with-java=1 \
- --with-js=1 \
- --with-go=1 \
- --with-rust=1 \
- --gold-dirs=testing/data/arrow-ipc-stream/integration/0.14.1 \
- --gold-dirs=testing/data/arrow-ipc-stream/integration/0.17.1 \
- --gold-dirs=testing/data/arrow-ipc-stream/integration/1.0.0-bigendian \
- --gold-dirs=testing/data/arrow-ipc-stream/integration/1.0.0-littleendian \
- --gold-dirs=testing/data/arrow-ipc-stream/integration/2.0.0-compression \
- --gold-dirs=testing/data/arrow-ipc-stream/integration/4.0.0-shareddict
+ run: conda run --no-capture-output ci/scripts/integration_arrow.sh $PWD /build
# test FFI against the C-Data interface exposed by pyarrow
pyarrow-integration-test: |
|
Something like that would be perfect. One small comment is it it would be good if we preserved the ability to override the tested languages, as in the past this has been very useful for temporarily disabling certain languages when they are having issues |
|
It makes sense. We can implement it by environment variables such as |
|
I've opted to temporarily disable the Java tests in order to get master green, and filed #4963 to track a better solution |
tustvold
changed the title
This was referenced Oct 23, 2023
kou
pushed a commit
to apache/arrow
that referenced
this pull request
Oct 25, 2023
…esting (#38403) ### Rationale for this change It is currently cumbersome to run integration tests from an other repository, as each tested implementation must be built explicitly. See explanatory comment at apache/arrow-rs#4957 (comment) ### What changes are included in this PR? Provide a wrapper script to automate building the implementations being tested. Allow usage of environment variables such as `ARROW_INTEGRATION_CPP`, etc. to selectively disable implementations. ### Are these changes tested? Yes, by construction. ### Are there any user-facing changes? No. * Closes: #38402 Authored-by: Antoine Pitrou <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
JerAguilon
pushed a commit
to JerAguilon/arrow
that referenced
this pull request
Oct 25, 2023
…tion testing (apache#38403) ### Rationale for this change It is currently cumbersome to run integration tests from an other repository, as each tested implementation must be built explicitly. See explanatory comment at apache/arrow-rs#4957 (comment) ### What changes are included in this PR? Provide a wrapper script to automate building the implementations being tested. Allow usage of environment variables such as `ARROW_INTEGRATION_CPP`, etc. to selectively disable implementations. ### Are these changes tested? Yes, by construction. ### Are there any user-facing changes? No. * Closes: apache#38402 Authored-by: Antoine Pitrou <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
alamb
pushed a commit
to alamb/arrow-rs
that referenced
this pull request
Nov 8, 2023
* Add additional integration test dependencies * Temporarily disable Java * Remove jpype
alamb
added a commit
that referenced
this pull request
Nov 8, 2023
* Temporarily Disable Java Integration Tests (#4957) * Add additional integration test dependencies * Temporarily disable Java * Remove jpype * Use new integration scripts (#4963) (#4988) --------- Co-authored-by: Raphael Taylor-Davies <[email protected]>
loicalleyne
pushed a commit
to loicalleyne/arrow
that referenced
this pull request
Nov 13, 2023
…tion testing (apache#38403) ### Rationale for this change It is currently cumbersome to run integration tests from an other repository, as each tested implementation must be built explicitly. See explanatory comment at apache/arrow-rs#4957 (comment) ### What changes are included in this PR? Provide a wrapper script to automate building the implementations being tested. Allow usage of environment variables such as `ARROW_INTEGRATION_CPP`, etc. to selectively disable implementations. ### Are these changes tested? Yes, by construction. ### Are there any user-facing changes? No. * Closes: apache#38402 Authored-by: Antoine Pitrou <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
dgreiss
pushed a commit
to dgreiss/arrow
that referenced
this pull request
Feb 19, 2024
…tion testing (apache#38403) ### Rationale for this change It is currently cumbersome to run integration tests from an other repository, as each tested implementation must be built explicitly. See explanatory comment at apache/arrow-rs#4957 (comment) ### What changes are included in this PR? Provide a wrapper script to automate building the implementations being tested. Allow usage of environment variables such as `ARROW_INTEGRATION_CPP`, etc. to selectively disable implementations. ### Are these changes tested? Yes, by construction. ### Are there any user-facing changes? No. * Closes: apache#38402 Authored-by: Antoine Pitrou <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #.
Rationale for this change
Seeing failures on master, this might fix it
https://github.com/apache/arrow-rs/actions/runs/6574785367/job/17860535457
What changes are included in this PR?
Are there any user-facing changes?